Papers with numeral systems
Scaling Behavior for Large Language Models regarding Numeral Systems: An Example using Pythia (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) are struggling with performing numeric operations accurately. |
| Approach: | They propose to use different numeral systems to scale different numerates in transformer-based large language models. |
| Outcome: | The proposed model is more data-efficient than base 10 and base 10 3 . the model is also more efficient on addition and multiplication . |